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MEMORANDUM  FOR  DIRECTOR,  DEFENSE  LOGISTICS  AGENCY 

SUBJECT:  Audit  Report  on  Defense  Logistics  Agency  Product  Quality  Deficiency 
Program  (Report  No.  98-063) 


We  are  providing  this  audit  report  for  information  and  use.  We  conducted  the 
audit  in  response  to  a  request  by  the  Defense  Logistics  Agency. 

Management  comments  on  a  draft  of  this  report  were  considered  in  preparing 
the  final  report.  The  comments  were  responsive  and  conformed  to  the  requirements  of 
DoD  Directive  7650.3;  therefore,  no  additional  comments  are  required. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Mr.  Terry  L.  McKinney,  Audit  Program  Director,  at 
(703)  604-9288  (DSN)  664-9288  or  Mr.  Michael  H.  Claypool,  Audit  Project  Manager, 
at  (703)  604-9291  (DSN  664-9291).  Respective  email  addresses  are 
<TMcKinney@dodig.osd.mil>  or  <MClaypool@dodig.osd.niil> .  See  Appendix  H 
for  the  report  distribution.  The  audit  team  members  are  listed  inside  the  back  cover. 

Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 
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Report  No.  98-063  February  5, 1998 

(Project  No.  7CF-0027) 

Defense  Logistics  Agency  Product  Quality  Deficiency  Program 


Executive  Summary 


Introduction.  We  initiated  the  audit  in  response  to  a  request  from  the  Director, 
Defense  Logistics  Agency.  He  was  interested  in  improving  the  product  quality 
deficiency  report  system,  which  is  used  by  DoD  Components  to  identify  and  purge 
nonconforming  material  from  inventory  and  to  provide  information  to  contracting 
officers,  contractors,  and  customers.  Nonconforming  material  is  a  defective  supply 
item  that  cannot  be  used  for  its  designed  purpose  because  of  material,  manufacturing, 
or  workmanship  defects.  The  product  quality  deficiency  program  enables  DoD 
customers  to  report  defects  to  &e  buying  commands,  who  then  investigate  the 
complaints,  exchange  deficiency  information  for  corrective  actions  through  the 
acquisition  and  support  process,  and  maintain  contractor  quality  history  to  be  used  for 
Government  best  value  contractor  source  selection.  From  October  1,  1995  through 
April  30,  1997,  17,051  product  quality  deficiencies  were  reported  to  the  Defense 
inventory  control  points  for  investigation  and  determination. 

Audit  Objectives.  The  overall  objective  of  the  audit  was  to  evaluate  the  product 
quality  deficiency  report  program  used  by  DoD  components;  however,  we  limited  the 
audit  to  include  only  the  Defense  Logistics  Agency  product  quality  deficiency  program. 
Specifically,  we  determined  whether  defective  products  were  reported  by  customers; 
and  if  reported,  whether  they  were  promptly  investigated  and  corrected.  We  also 
reviewed  DoD  progress  in  establishing  and  implementing  the  DoD- wide  Deficiency 
Reporting  System  Program.  We  also  evaluated  the  adequacy  of  management  controls 
applicable  to  the  product  quality  deficiency  report  program. 

Audit  Results.  The  Defense  Logistics  Agency  was  correct  in  assuming  there  were 
ways  to  improve  the  product  quality  deficiency  program.  Deficiency  reports  were 
initiated  when  nonco^orming  materials  were  identified,  and  investigations  into  the 
causes  of  the  deficiencies  were  promptly  conducted.  However,  collection  of  complaint 
information  needs  to  be  improved;  and  contractor  product  nonconformance  was  not 
always  used  in  evaluating  contractors’  past  performance.  These  two  conditions  warrant 
management  action. 

o  The  Defense  Logistics  Agency  product  quality  deficiency  investigations  did 
not  always  adequately  identify  the  cause  of  the  reported  product  deficiencies.  As  a 
result,  the  inventory  control  points  missed  opportunities  to  identify  contractors  with 
performance  problems,  and  improve  product  quality  (Finding  A). 
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o  The  Defense  Logistics  Agency  Automated  Best  Value  System  for  tracking 
contractor  past  performance  did  not  folly  reflect  contractor  quality  problems.  As  a 
result,  the  Defense  Logistics  Agency  increased  its  risk  of  procuring  products  from 
contractors  with  poor  past  performance  (Finding  B). 

Management  Controls.  The  management  control  program  could  be  improved.  We 
identified  a  material  weakness  applicable  to  the  audit  objectives  (Appendix  A). 

Summary  of  Recommendations.  We  recommend  that  the  Director,  Defense  Logistics 
Agency  require  that  the  DoD  Component  address  code  and  the  contractor  address  code 
be  a  mandatory  entry  to  close  a  product  quality  deficiency  investigation;  clarify  the  use 
of  the  “other/does  not  apply”  cause  code  for  customer  product  deficiency  complaints; 
eliminate  the  use  of  “special  inspection”  as  a  cause  code  for  customer  product 
deficiency  complaints;  and  direct  that  product  quality  deficiency  reporting  procedures 
ensure  that  the  Automated  Best  Value  System  includes  contractor  deficiencies  when 
rating  quality  for  past  performance. 

Management  Comments.  The  l!)efense  Logistics  Agency  concurred  with  all  of  the 
recommendations  and  indicated  that  corrective  actions  are  being  taken.  Management  is 
forming  a  Process  Action  Team  to  modify  the  customer  depot  complaint  system  and 
enhance  various  reporting  codes.  The  Process  Action  Team  is  scheduled  to  begin 
February  1998  and  complete  its  work  by  July  1998.  The  Defense  Logistics  Agency 
will  request  defense  inventory  control  points  to  submit  quarterly  reports  to  the  Materiel 
Management  Monthly  Management  Reviews  on  product  deficiency  cause  and 
correction,  as  well  as  disposition  codes  for  closed  investigations.  The  Defense 
Logistics  Agency  will  revise  the  Defense  Logistics  Agency  Directive  4155.2,  “Quality 
Assurance  Program  for  DLA  Inventory  Control  Points,”  to  incorporate  procedures  for 
contractor  notification  related  to  the  Automated  Best  Value  System.  Also,  management 
agreed  to  establish  a  single  quality  rating  for  the  Automated  Best  Value  System  to 
measure  both  contractor  product  quality  deficiencies  and  product  laboratory  test 
failures.  This  procedure  has  been  implemented  and  will  be  initiated  in  February  1998. 
See  Part  I  for  a  discussion  of  management  comments,  and  Part  HI  for  a  complete  text 
of  management  comments. 

Audit  Response.  The  Defense  Logistics  Agency  comments  are  folly  responsive  to  the 
reconunendations.  We  commend  the  Defense  Logistics  Agency  for  seekmg  to  improve 
its  processes  and  for  implementing  the  initiative  to  establish  a  single  quality  rating  for 
the  Automated  Best  Value  System  for  contractor  product  quality  deficiencies  and 
laboratory  test  failures  so  promptly. 
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Audit  Background 


The  audit  was  initiated  at  the  request  of  the  Director,  Defense  Logistics  Agency 
(DLA),  in  the  interest  of  improving  the  product  quality  deficiency  reportiujg 
system.  The  Defense  Logistics  Agency  Directive  4155.24.  “Product  Quality 
Deficiency  Report  Program,”  May  1,  1997  (DRAFT),  states  that  the  program  is 
designed  to  identify  and  purge  nonconforming  products  from  inventory  and  to 
provide  feedback  to  contracting  officers,  contractors,  and  DoD  Component 
customers.  The  Product  Quality  Deficiency  Report  (PQDR)  program  enables 
Components  to  exchange  information  about  necessary  corrective  actions 
throughout  the  acquisition  and  support  process,  and  to  maintain  a  history  of 
contractor  quality.  In  addition,  the  program  provides  the  initial  reporting, 
cause,  correction,  and  status  of  customer  product  quality  deficiencies.  The 
guidance  states  that  the  investigative  data  gathered  from  the  PQDR  program 
should  be  used  to  identify  problems,  trends,  and  recurring  deficiencies  detected 
on  new  or  newly  reworked  Government-owned  products.  The  key  program 
participants  include  the  DoD  Component  deficiency  report  originator,  screening 
point,  and  the  action  point. 

The  PQDR  program  separates  deficiency  reports  into  two  categories:  Category 
I  and  Category  II.  A  Category  I  is  defined  as  a  product  quality  deficiency 
which  could: 

o  cause  death,  injury,  or  severe  occupational  illness;  could  cause  loss  or 
major  damage  to  a  weapons  system; 

o  critically  restrict  the  combat  readiness  capabilities  of  a  Component 
organization;  or 

o  result  in  a  production  line  stoppage.  A  product  quality  deficiency 
which  does  not  meet  the  Category  I  criteria  is  classified  as  a  Category  II 
deficiency  report. 

For  FY  1996  and  the  first  3  quarters  of  FY  1997,  the  four  DLA  supply  centers 
had  non-fuel  material  sales  of  $7.3  billion  to  DoD  Components  and  $5.5  billion 
to  other  Federal  agencies.  During  October  1,  1995  through  April  30,  1997,  the 
DLA  Inventory  Control  Points  (ICPs)  recorded  17,051  product  deficiency 
reports  for  investigation  and  determination  of  the  complaint.  These  reports  do 
not  include  deficiency  complaints  the  military  Services  receive  and  investigate 
for  material  that  they  manage  through  their  own  supply  center  ICPs.  Our 
review  focused  only  on  complaints  processed  by  DLA. 

Customer  Depot  Complaint  System.  The  Customer  Depot  Complaint  System 
(the  System)  records  and  processes  customer  and  depot  deficiency  complaints. 

A  complaint  refers  not  just  to  PQDRs  but  to  any  type  of  deficiency  in  &e 
quality  of  products  issued,  stored,  or  used  by  DoD  customers.  The  System  is 
designed  to: 
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o  record  the  status  and  aging  of  complaints  received  at  the  DLA ICP; 

o  provide  information  for  management  analysis; 

o  provide  interface  with  the  Automated  Discrepancy  Reporting  System 
located  at  Defense  Distribution  System  depots;  and 

o  automate  the  routine  tasks  involved  with  the  processing  of  deficiency 
complaints  received  at  the  ICP. 

DLA  Handbook  4140.4,  “Customer  Depot  Complaint  System,”  April  1,  1996, 
assigns  responsibility  for  system  administration  to  the  Director  of  Supply 
Operations  along  with  responsibility  for  analysis  of  the  completeness,  accuracy, 
and  use  of  the  System.  Tlie  Director  is  also  charged  with  performing  trend 
analysis  for  complaints,  including  disposition  of  closed  complaint  results,  and  to 
report  adverse  or  significant  trends  to  management. 

Contractor  Best  Value.  As  a  result  of  the  Packard  Commission’s 
recommendation  to  use  commercial  buying  practices,  the  DLA  inventory  control 
points  implemented  “best  value”  programs  to  rate  conunercial  contractors  for 
procurements.  Contractor  best  value  uses  a  comparative  assessment  of  a 
contractor’s  offer  on  the  basis  of  factors  other  than  price.  A  contractor’s  past 
performance  is  one  factor  for  assessing  the  performance  risk  of  future  contracts. 
In  1996,  DLA  established  the  Automated  Best  Value  System  (ABVS)  as  a 
means  to  capture  contractor  past  performance  in  order  to  assess  the  procurement 
risk  associated  with  each  potential  contractor.  The  ABVS  collects  contractor 
past  performance  data  and  calculates  a  numeric  score  used  by  contracting 
officers  when  evaluating  contractor  proposals.  The  primary  source  of  past 
performance  is  the  System  that  provides  information  for  contractor  product 
nonconformance.  Contract  awards  using  ABVS  consider  contractor  price, 
quoted  delivery,  and  performance  risk. 

Deficiency  Reporting  System.  In  1992,  the  Joint  Logistics  Systems  Center 
initiated  the  development  of  the  DoD-wide  Deficiency  Reporting  System 
Program  to  provide  a  software  system  for  DoD  component  users  to  originate 
and  resolve  a  product  quality  complaint. 

In  October  1996,  the  Assistant  Deputy  Under  Secretary  of  Defense  (Logistics 
Business  Systems  and  Technology  Development)  issued  a  memorandum 
terminating  all  future  software  development  and  deployment  activities  for  the 
DoD-wide  Deficiency  Reporting  System  Program.  Appendix  C  includes 
additional  information  on  this  program. 
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Audit  Objectives 


The  primary  audit  objective  was  to  determine  whether  defective  products  were 
being  reported  by  customers,  and  if  reported,  whether  they  were  promptly 
investigated  and  corrected.  The  audit  was  to  evaluate  the  product  quality 
deficiency  report  programs  used  by  DoD  components;  however,  we  limited  the 
audit  to  include  oriy  the  Defense  Logistics  Agency  product  quality  deficiency 
report  program.  We  also  reviewed  the  DoD-wide  Deficiency  Reporting  System 
Program  to  determine  if  it  was  providing  managers  with  timely  information 
needed  to  identify  inferior  products.  We  also  evaluated  the  adequacy  of  the 
management  control  program  as  it  applied  to  the  primary  audit  objective.  See 
Appendix  A  for  a  discussion  of  the  audit  process  and  a  review  of  the 
management  control  program.  Appendix  B  has  a  summary  of  prior  audit 
coverage  related  to  the  audit  objectives. 
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Finding  A.  Quality  Deficiency  Reporting 

Defense  Logistics  Agency  product  quality  deficiency  investigations  did 
not  always  identify  Ae  causes  of  reported  product  deficiencies.  This 
occurred  because  the  inventory  control  points  were  not  provided 
complete  product  quality  deficiency  report  data,  and  did  not  correctly 
code  the  cause  of  deficiencies  or  results  of  the  investigations.  As  a 
result,  the  Government  missed  opportunities  to  identify  contractors  with 
performance  problems  and  improve  product  quality. 


Deficiency  Complaint  System 


DLA  uses  the  Customer  Depot  Complaint  System  (the  System)  to  record  and 
track  a  product  deficiency  complaint  until  resolution  of  &e  investigation. 

The  principal  participants  in  the  PQDR  process  are  the  originator,  originating 
point,  screening  point,  and  action  point. 

o  An  originator  is  a  user  who  discovers  the  defective  product  and 
initiates  the  deficiency  report. 

o  An  originating  point  is  an  organization  within  a  Component  (or  a 
contractor  who  receives  a  Government  product)  that  finds  a  product  quality 
deficiency  and  reports  it  to  the  designated  Component  screening  point. 

o  A  screening  point  is  a  designated  activity  identified  within  each 
Component  that:  reviews  the  PQDR  for  proper  categorization,  validity, 
correctness  of  entries,  accuracy,  and  completion  of  information  addresses; 
determines  and  transmits  the  PQDR  to  the  proper  action  point  within  or  outside 
the  Component,  for  example  an  ICP;  maintains  an  audit  trail  for  each  PQDR; 
reviews  closed-out  responses  fi’om  action  points;  and  collects,  maintains,  and 
exchanges  PQDR  data  with  other  DoD  Components. 

The  PQDR  investigation  begins  at  the  ICP  action  point.  The  Quality  Assurance 
Specialist  (QAS)  is  primarily  responsible  for  managing  investigations  and 
performs  Ae  majority  of  the  action  point  responsibilities.  During  the 
investigation,  the  QAS  enters  codes  and  text  to  explain  the  cause  of  toe 
deficiency,  toe  action  taken  to  correct  toe  deficiency,  and  toe  disposition  of  toe 
defective  product. 
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Finding  A.  Quality  Deficiency  Reporting 


Action  point  responsibilities  include  determining: 

o  whether  the  complaint  should  be  investigated  and  the  scope  of  the 
investigation; 

o  the  cause  of  the  deficiency,  the  corrective  action,  and  the  disposition 
of  the  defective  product; 

o  whether  the  contractor  should  repair,  replace,  or  provide 
consideration  to  the  Government  for  the  defective  product; 

0  whether  to  examine  like  products  in  inventory;  and 

o  whether  to  put  the  inventory  on  hold  until  the  ICP  completes  the 
investigation. 

An  action  point  decision  to  examine  items  in  inventory  may  involve  thousands 
of  items  and  result  in  defective  products  being  returned  to  a  Defense  depot  or  a 
contractor  for  disposal  or  repair.  Only  an  action  point  is  authorized  to  inform 
other  Components  of  deficiencies.  Appendix  D  reviews  additional  action  point 
responsibilities. 


Customer  Complaint  Information 


Product  quality  deficiency  investigations  did  not  always  adequately  identify  the 
cause  of  the  reported  product  deficiencies.  Missing  data  and  vague  codes  for 
explaining  the  cause  of  the  deficiencies  limited  the  usefulness  of  the  System 
information. 

Missing  Data.  The  DLA  ICPs  provided  17,051  deficiency  records.  The 
records  included: 

o  282  PQDR  category  1  reports; 

o  15,297  PQDR  category  2  reports;  and 

o  1,472  reports  on  failed  laboratory  tests. 

Analysis  of  the  records  showed  the  contractor  Commercial  and  Government 
Entity  code  (CAGE),  contract  number,  and  Department  of  Defense  Address 
Activity  Code  (DODAAC)  record  fields  had  missing  data.  Figure  1  shows  that 
approximately  44  percent  of  the  Cage  codes,  28  percent  of  the  contract 
numbers,  and  17  percent  of  the  DODAACs  were  missing. 
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Finding  A.  Quality  Deficiency  Reporting 


Figure  1.  Missing  Data  for  Selected  CDCS  Data  Fields 


CAGE  Code 


Contract  Number 


DODAAC 


■  Missing 
13  included 


The  action  point  needs  the  CAGE  code  number  in  order  to  identify  the 
contractor,  initiate  recoupment  action,  supply  the  data  to  the  Automated  Best 
Value  System,  and/or  get  the  contractor  to  support  an  investigation  into  the 
cause  of  the  defect.  Also,  the  contract  number  and  CAGE  code  can  be  used  to 
facilitate  a  stock  screening.  The  CAGE  code  can  be  identified  from  information 
contained  in  the  contract.  Therefore,  it  is  important  that  the  contract  number  be 
available  to  the  action  point.  The  activity  address  code  is  also  necessary  as  it 
furnishes  information  on  the  source  of  the  deficiency  report.  Using  the  activity 
address  code  allows  ICPs  to  analyze  the  source  of  the  deficiency  report  to 
determine  if  large  numbers  of  deficient  products  are  being  received  by  specific 
Component  customers. 

The  DoD  Joint  Service  regulation  governing  the  PQDR  process  requires  that  the 
originating  and  screening  points  provide  each  of  these  data  elements.  However, 
if  the  PQDR  includes  the  contract  number,  the  action  point  can  identify  the 
CAGE  code  from  the  contact  or  the  DOD  ACC  by  calling  the  originator  of  the 
product  deficiency  complaint. 

Adding  Missing  Data.  The  originating  point  and  screening  points  did  not 
provide  complete  information  for  the  PQDR.  During  the  investigation,  the 
action  point  should  have  added  the  missing  CAGE  codes,  DODAAC,  and 
contract  numbers.  Of  the  7,439  System  records  with  missing  CAGE  codes,  the 
QAS  could  have  identified  the  missing  CAGE  codes  for  2,659  System  records 
by  using  the  contract  number  already  available  to  them.  Also,  the  originating 
point  DODAAC  information  could  be  acquired  from  the  component’s  screening 
point. 


Product  Deficiency  Cause  Codes 


A  major  objective  of  a  PQDR  investigation  is  to  identify  the  root  cause  of  the 
deficiency.  For  75  percent  of  the  records  examined,  the  ICPs  used  two  cause 
codes  to  describe  the  results  of  the  investigations;  “other/does  not  apply” 
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(7,691  records)  and  “special  inspection”  (4,855  records).  As  a  result,  neither 
&e  cause  of  the  product  defect  nor  the  party  at  fault  (the  contractor  or  the 
Government)  could  be  determined.  Figure  2  shows  Ae  distribution  of  cause 
codes  for  16,898  records. 


Figure  2.  Distribution  of  Cause  Codes 


ICP  use  of  cause  codes  and  the  results  of  the  product  quality  deficiency 
investigation  varied  by  DLA  supply  center.  For  example,  die  Defense 
Personnel  Support  Center  used  other/does  not  apply  for  8.29  percent  of  its  cause 
codes;  while  the  Defense  Supply  Center,  Columbus  used  other/does  not  apply 
for  67.83  percent  of  its  cause  codes.  The  Defense  Supply  Center,  Richmond 
used  special  inspection  for  2.28  percent  of  its  cause  codes  while  the  Defense 
Personnel  Support  Center  used  it  for  88.78  percent  of  its  cause  codes. 

Appendix  G  shows  the  variation  by  ICP  Supply  Center. 

Other/does  not  apply  Cause  Codes.  The  DLA  guidance  on  the  use  of  the 
other/does  not  apply  cause  code  is  ambiguous.  In  1988  DLA  instructed  ICPs  to 
use  the  other/does  not  apply  cause  code  exclusively  for  product  deficiency 
complaints  that  the  ICP  determined  should  not  have  been  submitted  because 
they  were  invalid  complaints.  During  our  audit,  DLA  Headquarters 
reemphasized  this  policy  to  ICPs.  However,  the  System  manual,  published  in 
1996,  contains  the  cause  codes  for  ICPs  to  use,  and  does  not  state  that  the  use  of 
other/does  not  apply  should  be  limited  to  invalid  customer  complaints. 

Reason  for  Use  of  Other/does  not  apply  Cause  Code.  According  to 
the  ICP  staffs,  a  large  number  of  records  were  closed  as  other/does  not  apply 
primarily  because  the  existing  System  cause  codes  were  not  adequate  to  describe 
all  of  the  deficiency  causes. 

For  example,  the  Defense  Supply  Center,  Columbus  used  the  other/does  not 
apply  cause  code  for  Component  customer  credit  for  defective  products 
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purchased  from  DLA  .  The  Defense  Supply  Center,  Columbus  used  this  cause 
code  to  record  customer  credit  for  defective  material  and  established  1,072 
records  for  customer  credit. 

Two  of  three  ICPs  had  requested  other  deficiency  codes  for  the  System.  DLA 
deferred  the  requests,  because  the  DoD-wide  Deficiency  Reporting  System 
Program,  which  would  have  new  codes,  was  going  to  replace  the  current 
System.  However,  DLA  is  not  implementing  this  system.  Appendix  C 
discusses  the  Deficiency  Reporting  System  Program. 

Adherence  to  DLA  Guidance.  DLA  policy  required  ICPs  to  use  the 
other/does  not  apply  cause  code  only  for  invalid  customer  product  deficiency 
complaints.  To  measure  compliance  with  that  policy,  we  analyzed  the 
disposition  codes  for  the  records  coded  other/does  not  apply  as  the  cause  of  the 
deficiency.  The  correct  disposition  code  used  for  other/does  not  apply 
complaints  should  have  always  been  coded  invalid  complaint.  Figure  3  shows 
that  the  ICPs  did  not  follow  DLA  policy  because  86  percent  of  the  records  had  a 
disposition  code  that  was  not  coded  invalid  complaint. 


Figure  3.  Disposition  Codes  for  Other/Does  Not  Apply  Cause  Codes 


All  other 

Contractor  21% 


Does  not  apply 
13% 


Disposal 

38% 


An  example  of  the  disposition  of  other/does  not  apply  cause  code  are  PQDRs 
submitted  by  3  different  Navy  activities  against  an  item  in  Federal  stock  class 
4730,  “Hose,  Pipe,  Tube,  Lubrication,  and  Railing  Fitting.”  The  ICP  received 
11  PQDRs  against  this  item  for  the  period  covered  by  our  System  database. 

The  ISP  action  point  coded  10  of  11  PQDR  deficiency  causes  as  other/does  not 
apply  and  the  remaining  PQDR  was  coded  as  a  contractor  product 
nonconformance.  The  action  point  assigned  3  different  disposition  codes  to  the 
11  PQDRs.  For  the  10  PQDRs  coded  other/does  not  apply,  the  investigator 
coded  6  items  to  return  to  depot  inventory  and  coded  4  items  to  authorize 
disposal  of  the  defective  items.  For  the  one  PQDR  coded  contractor  product 
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nonconformance,  the  defective  product  was  returned  to  the  contractor.  Disposal 
of  the  4  items  certainly  implies  the  items  were  defective  products,  yet  the 
PQDRs  did  not  have  a  correct  deficiency  cause  code. 

Special  Inspection  Cause  Code.  The  ICPs  coded  29  percent  of  the  deficiency 
complaints  as  special  inspection.  The  special  inspection  cause  code  describes  an 
action  taken  instead  of  a  cause  for  the  product  deficiency.  The  special 
inspection  codes  were  being  used  primarily  to  record  customer  credits  for 
defective  products  purchased  from  DLA.  The  Defense  Personnel  Support 
Center  accounted  for  78  percent  of  the  total  special  inspection  records  (4,855). 
This  was  a  result  of  the  center’s  policy  that  required  a  PQDR  with  fewer  than 
5  defective  items  and  with  a  total  cost  of  less  than  $250  to  be  coded  special 
inspection.  The  center  developed  this  policy  to  grant  credits  to  customers  for 
defective  products. 


Metrics 


Metrics  Requirements.  DLA  headquarters  required  the  ICPs  to  report 
quarterly  metrics  on  the  number  of  PQDRs  received,  closed,  open  for 
investigation,  and  the  average  processing  time.  These  metrics  focused  on  the 
efficiency  of  processing  and  investigating  PQDRs  instead  of  the  causes  of  the 
deficiencies  being  submitted  and  the  overall  outcome  of  the  investigations. 

The  ICPs  were  not  reporting  metrics  for  the  root  cause  of  the  product 
deficiencies,  and  disposition  of  the  investigations.  A  March  31,  1993,  DLA 
Deputy  Executive  Director  (Quality  Assurance)  letter,  “Product  Quality 
Deficiency  Report  (PQDR)  Program,”  to  Commander  of  DLA  Supply  Centers 
provided  guidance  for  improving  the  PQDR  investigative  process.  The  Director 
said,  “Program  effectiveness  should  also  be  determined  by  the  establishment  of 
metrics  which  measure  PQDR  investigations  and  resolutions  related  to: 
identifying  the  root  cause  of  the  deficiency;  introduction  of  corrective  actions  to 
prevent  recurrence;  satisfying  the  complaint  initiator;  recoupment  from 
contractors  for  contractor-caused  deficiencies;  as  well  as  times  associated  with 
the  accomplishing  of  these  process  objectives.”  Unfortunately,  Headquarters 
DLA  and  the  ICft  use  only  metrics  associated  with  timeliness. 

Metrics  Usage.  People  in  organizations  usually  adapt  their  behavior  to  what 
management  is  measuring,  that  is,  those  things  that  management  measures 
receive  the  most  attention  from  employees.  When  DLA  emphasized  PQDR 
closing  times  and  the  inventory  of  open  customer  deficiency  complaints, 
average  processing  time  and  the  number  of  open  complaints  decreased.  Metrics 
that  focus  on  measuring  PQDR  program  effectiveness  should  improve  the 
investigation  and  reporting  of  the  results  of  deficiency  investigations.  The 
number  of  investigations  for  which  the  deficiency  cause  was  “other/does  not 
apply”  will  decrease  if  the  ICPs  have  to  report  to  management  why  “other/does 
not  apply”  was  used  in  almost  one  half  of  the  System  records  we  reviewed. 
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Emphasis  on  determining  the  actual  cause  of  deficiencies  is  the  first  step  in 
correcting  deficiencies,  reducing  product  inventory  costs,  and  assuring  customer 
satisfaction  of  DLA  products. 


Realizing  System  Benefits 


The  inventory  control  points  missed  opportunities  to  identify  contractors  with 
performance  problems,  and  improve  product  quality. 

Contractor  Past  Performance.  PQDRs  are  the  primary  source  of  information 
about  quality  of  contractor  past  performance.  ABVS  uses  past  performance 
quality  data  to  rate  contractors.  When  the  CAGE  code  is  missing,  the  ABVS 
cannot  rate  the  contractor’s  quality  performance.  For  example,  187  of  1,833 
records  coded  as  contractor  nonconformance  with  the  contract  number  did  not 
have  the  CAGE  code.  Thus,  10  percent  of  the  records  coded  as  a  contractor 
deficiency  may  not  have  used  to  calculate  ABVS  quality  ratings.  This  missing 
information  could  have  been  the  deciding  factor  in  a  contract  award. 

Improving  Product  Quality.  When  the  PQDR  investigation  did  not  identify 
the  cause  of  the  product  deficiency,  DLA  lost  opportunities  to  improve  the 
quality  of  future  procurements.  The  PQDR  program  should  identify  the  reason 
&at  contractors  deliver  defective  products  to  the  Government.  Either  the 
Government  or  the  contractor  was  the  cause  of  the  deficiency.  When  the 
investigation  did  not  determine  the  root  cause  of  the  deficiency  and  the 
responsible  party,  DLA  could  not  initiate  action  to  correct  the  deficiency. 

When  DLA  did  not  identify  the  responsible  contractor  or  establish  that  the 
contractor  was  at  fault  for  a  defective  product,  the  Government  could  not  obtain 
the  repair,  replacement,  or  financial  consideration  for  the  defective  product 
from  die  contractor.  The  Government  absorbed  the  entire  cost  for  disposing  of 
and  replacing  defective  products. 

During  October  1,  1995  through  March  31,  1997,  ICPs  identified  $26.1  million 
of  contractor  defective  products.  From  this  total,  contractors  will  provide 
consideration  valued  at  $16.8  million  (repair,  replace,  or  financial 
consideration)  for  the  defective  product. 

Obtaining  Value.  Because  the  purpose  of  a  PQDR  investigation  includes 
identifying  the  cause  of  the  deficiency  and  75  percent, of  the  investigations  did 
not  identify  the  cause,  ICPs  were  not  using  resources  effectively  when 
investigating  PQDRs.  According  to  an  October  1994,  DLA  study,  “Cost  of 
Nonconforming  Supplies  Update,”  the  average  administrative  cost  for  actions 
that  encompass  quality  deficiency  report  processing,  investigation,  and 
resolution  was  $868  per  complaint  for  a  typical  DLA-managed  item.  The 
analysis  also  showed  that  the  average  holding  cost  per  quality  deficiency  report 
was  5.98  percent  of  the  contract  value  for  a  typical  item.  Holding  cost  is  the 
sum  of  two  elements.  The  first  is  opportunity  cost  or  the  value  of  money  for 
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the  product  in  storage  that  is  restricted  from  shipment  pending  the  outcome  of 
the  investigation.  The  second  is  supply  cost  or  the  cost  of  holding  the  physical 
inventory  in  a  warehouse. 

The  study  stated  that  the  sum  of  the  administrative  and  holding  costs  represents 
a  “minimum”  total  complaint  cost.  Using  the  study’s  cost  of  $868  per 
complaint,  the  cost  of  processing  the  7,691  PQDRs  coded  “other”  was 
$6,675,788.  This  estimate  does  not  include  the  holding  costs  associated  with  a 
deficiency  complaint.  The  best  way  to  obtain  value  is  to  identify  and  correct  the 
cause  of  the  defect,  and  when  applicable,  receive  consideration  from  the 
contractor  for  nonconforming  products. 


Conclusion 


DLA  was  correct  in  assuming  that  the  PQDR  system  needed  improvement. 
Missing  data,  inconclusive  results  for  investigations,  and  reporting  of  metric 
information  that  focuses  on  the  efficiency  of  the  process  all  result  in  less  than 
optimum  process  results.  Missing  data  needed  to  identify  the  contractor  affects 
the  ability  to  recover  consideration  from  the  contractor  when  the  contractor 
caused  the  deficiency.  Missing  data  also  affects  the  contractor  ratings  in  the 
ABVS.  The  use  of  cause  codes  that  do  not  provide  useful  information  about  the 
reason  for  product  deficiencies  hampers  efforts  to  improve  product  quality  for 
customers.  Inconclusive  product  deficiency  cause  codes  deny  nianagement 
information  to  focus  attention  on  product  quality  problems  needing  corrective 
action  by  the  Government.  DLA  directs  significant  attention  toward  improving 
contractor  performance  through  the  use  of  ABVS  to  rate  contractor  quality  past 
performance.  Yet,  the  System  records  we  reviewed  had  only  about  11  percent 
of  the  PQDRs  as  contractor-caused  deficiencies. 


Recommendations  and  Management  Comments 


A.  We  recommend  that  the  Director,  Defense  Logistics  Agency: 

1.  Modify  the  customer  depot  complaint  system  to  require  the 
Commercial  and  Government  Entity  code  as  a  mandatory  entry  for  closing 
a  complaint  whenever  the  record  includes  the  contract  number. 

2.  Modify  the  customer  depot  complaint  system  to  require  the  DoD 
Activity  Address  Code  as  a  mandatory  entry  for  closing  a  product  quality 
deffciency  investigation. 

3.  Clarify  proper  use  of  the  other/does  not  apply  cause  code  for 
customer  deficiency  complaints. 
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4.  Eliminate  the  use  of  special  inspection  as  a  cause  code  for 
customer  deflciency  complaints. 

5.  Revise  the  customer  depot  complaint  system’s  cause,  disposition, 
and  action  codes  and  replace  them  with  more  descriptive  codes  for 
deficiency  complaints. 

6.  Require  the  defense  inventory  control  points  to  report  quarterly 
metrics  on  product  quality  deHciency  cause,  correction,  and  disposition 
codes  for  closed  investigations.  The  report  should  also  include  information 
on  initially  missing  contract  numbers.  Commercial  and  Government  Entity 
codes,  and  DoD  Address  Activity  Codes. 

Defense  Logistics  Agency,  Conunents.  DLA  concurred  with  all  of  the 
recommendations  and  stated  that  a  Process  Action  Team  (PAT)  will  be  initiated 
in  February  1998  to  address  the  recommendations  for  corrective  actions.  DLA 
provided  a  timetable  to  develop  draft  codes,  metrics,  and  policy.  DLA  will 
request  that  the  Defense  Inventory  Control  Points  report  quarterly  through  the 
Materiel  Management  Monthly  Management  Reviews  on  product  deficiency 
causes,  corrections,  and  disposition  codes  for  closed  investigations.  DLA  also 
stated  that  the  agency  policy  in  Defense  Logistics  Agency  Directive  (DLAD) 
4155.2,  “Quality  Assurance  Program  for  DLA  Inventory  Control  Points,”  will 
be  revised  to  include  these  recommended  changes. 
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The  Automated  Best  Value  System  for  contractor  past  performance  did 
not  fully  reflect  contractor  quality  problems.  This  occurred  because  the 
Defense  Logistics  Agency  did  not  ensure  that  the  data  inputs  to  the 
system  were  accurate  and  complete  and  that  contractors  were  penalized 
equally  for  nonconforming  products.  As  a  result,  the  Defense  Logistics 
Agency  increased  its  risk  of  procuring  products  from  contractors  with 
poor  past  performance. 


Automated  Best  Value  System  Methodology 


Because  of  the  Packard  Commission  recommendation  to  use  commercial  buying 
practices,  DLA  implemented  “Best  Value”  programs  to  rate  contractors  for 
future  procurements.  DLA  defines  “Best  Value”  as  a  comparative  assessment 
of  offers  on  the  basis  of  factors  in  addition  to  price.  DLA  evaluates  past 
performance  as  one  means  of  assessing  contractor  performance  risk. 

DLA  uses  the  Automated  Best  Value  System  (ABVS)  to  record  past 
performance  of  active  contractors.  The  ABVS  is  an  automated  system  which 
collects  and  analyzes  contractor  past  performance  data  from  other  DLA 
automated  systems,  including  the  customer  depot  complaint  system.  The  ABVS 
translates  the  data  into  a  numeric  rating  that  a  contracting  officer  can  use  to 
evaluate  a  contractor. 

The  ABVS  rates  contractors  on  a  100  point  system,  consisting  of  50  points  for 
quality  and  50  points  for  delivery  over  the  preceding  12  months.  Contractors 
with  high  ratings  represent  a  lower  performance  risk.  Each  contractor  receives 
a  rating  for  each  Federal  stock  class  in  which  the  contractor  provides  the 
Government  products.  In  addition,  DLA  provides  contractors  with  an  overall 
rating  that  is  a  summary  of  their  performance  in  all  supply  classes.  Under 
current  procedures,  DLA  must  notify  a  contractor  of  a  quality  deficiency 
problem  before  deducting  points  from  the  contractor  numeric  rating.  Then  the 
ABVS  deducts  points  from  the  contractor’s  rating  based  on  the  point 
assignments  as  described  in  figure  4. 
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Figure  4.  Automated  Best  Value  System  Ratings 

Quality:  (50  points) 

Laboratory  Tests  Failures 

15  points 

Product  Complaints  (PQDRs) 

25  points 

Packaging  Complaints 

10  points 

Delivery:  (50  points) 

Frequency  of  late  contract  lines 

30  points 

Average  number  of  days  late 

20  points 

If  a  contractor  has  no  deficiencies  the  contractor  receives  100  points  for  past 
performance.  The  ABVS  deducts  product  quality  points  for  product  complaints 
(PQDRs),  packaging  complaints,  and  laboratory  test  failures.  The  ABVS 
deducts  delivery  points  based  on  the  frequency  of  delinquent  contract  lines  and 
the  number  of  days  late.  The  rating  points  are  deducted  from  the  contractor 
when  deficiencies  occur.  The  ABVS  prorates  the  deduction  of  points  based  on 
the  number  of  deficiencies  against  the  number  of  contract  lines  awarded.  For 
example,  if  a  contractor  has  one  PQDR  and  one  contract  line,  the  System  would 
deduct  the  full  25  points  for  product  complaints.  If  a  contractor  has  one  PQDR 
and  two  contract  lines,  the  ABVS  would  deduct  12.5  points  for  product 
complaints.  For  laboratory  test  failures,  ABVS  prorates  15  points  based  on  the 
parts  that  failed  divided  by  parts  tested. 


Inaccurate  and  Incomplete  Quality  Data 


The  ABVS  did  not  fiilly  reflect  contractor  quality  problems  because  ICPs  did 
not  ensure  accuracy  and  completeness  of  the  data.  As  a  result,  the  ABVS 
quality  ratings  did  not  accurately  rate  contractors  for  past  performance. 

Product  Quality  Deficiency  Reports.  The  ABVS  did  not  make  point 
deductions  for  product  complaints  recorded  in  the  System.  We  identified 
30  contractors  with  3  or  more  PQDRs  issued  against  a  Federal  stock  class 
number  where  DLA  determined  the  contractor  was  at  fault.  For  seven 
contractors,  ABVS  did  not  list  the  PQDRs  issued  against  the  contractors  for  the 
specific  product  Federal  stock  class.  For  three  contractors,  ABVS  did  not  list 
the  PQDRs  for  the  contractor  overall  rating  for  the  specific  product  Federal 
stock  class  number.  For  the  remaining  20  contractors,  ABVS  listed  the  PQDRs 
against  the  product  Federal  stock  class  number  and  the  overall  contractor  rating; 
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however,  the  contractors’  quality  ratings  did  not  reflect  all  of  the  multiple 
PQDRs  listed  for  the  product  Federal  stock  class.  Accordingly,  ABVS  quality 
ratings  for  multiple  PQDRs  did  not  completely  reflect  the  contractors  past 
performance  ratings. 

Test  Failures.  ABVS  did  not  make  point  deductions  for  test  failures 
recorded  in  the  customer  depot  complaint  system.  We  identified  1 10  laboratory 
product  test  failures  caused  by  a  contractor  deficiency  that  meet  the  ABVS 
rating  criteria.  Of  the  110  tests,  we  identified  9  test  failures  that  ABVS  did  not 
list  for  the  contractor’s  product  Federal  stock  class  number.  ABVS  recorded 
27  test  failures  but  failed  to  deduct  points  from  the  contractor  performance 
rating  against  the  Federal  stock  class  number.  ABVS  deducted  points  for  the 
remaining  74  test  failures  and  35  of  the  contractors  received  12  points  or  more 
of  the  possible  15  points  for  product  test  failures.  Four  of  these  contractors  had 
several  test  failures  for  a  product  Federal  stock  class  number,  yet  they  still 
received  high  ratings  and  remained  strong  contenders  for  future  business.  For 
example,  one  contractor  had  three  test  failures  but  ABVS  rated  the  contractor  at 
92.8  points  for  the  product  Federal  stock  class  number. 

Cause  of  Inaccurate  and  Incomplete  Data.  ICPs  did  not  ensure  that 
inputs  to  the  System,  which  updates  the  ABVS,  were  accurate  or  complete. 
Accordingly,  ABVS  could  not  record  accurate  contractor  quality  ratings  and  this 
accounted  for  the  majority  of  the  incomplete  and  inaccurate  ratings  for 
contractor  past  performance. 

ICPs  also  excluded  PQDRs  and  laboratory  test  failures  from  ABVS  quality 
ratings  for  other  reasons.  For  example,  quality  assurance  specialists  will 
exclude  data  when  there  is  a  question  concerning  the  PQDR  or  test  failure 
investigation  results  on  the  cause  of  the  product  deficiency,  or  when  contractors 
successfully  challenge  the  Government  on  whether  the  product  deficiency  was 
caused  by  the  contractor. 

Under  current  procedures,  ICPs  must  notify  a  contractor  of  a  product  quality 
problem  prior  to  subtracting  points  from  the  contractor’s  numeric  rating.  For 
example,  contractors  selling  to  the  Defense  Supply  Center,  Richmond 
successfully  challenged  the  ABVS  quality  rating  55  times  over  the  last 
two  years.  Forty  six  of  the  55  contractors  were  successful  in  reversing  quality 
rating  deductions  because  the  contractors  were  not  notified.  Quality  assurance 
specialists  need  to  notify  contractors  of  all  deficiencies  so  that  past  performance 
quality  data  can  be  used  in  ABVS  to  rate  contractors  for  future  procurements. 

Increased  Risk.  As  a  result  of  inaccurate  ABVS  quality  ratings,  DLA 
increased  its  risk  of  purchasing  products  from  contractors  with  poor  past 
performance.  During  contractor  selection,  ICPs  use  the  ABVS  ratings  to 
compare  the  past  performance  of  possible  contractors.  Since  price  and  past 
performance  are  the  primary  evaluation  factors  used  to  award  a  contract,  correct 
contractor  ratings  are  important  to  assist  contracting  officers  in  making  good 
contract  award  decisions. 
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Unequal  Allocation  of  Quality  Rating  Points 


DLA  did  not  penalize  contractors  equally  for  nonconforming  products.  This 
occurred  because  DLA  did  not  modify  the  ABVS  to  coincide  with  DLA  changes 
to  the  laboratory  testing  procedures. 

Development  of  the  Automated  Best  Value  System.  When  DLA  developed 
the  ABVS,  a  quality  rating  of  40  points  was  used  to  measure  contractor  quality 
deficiencies  (15  points  for  laboratory  test  failures  and  25  points  for  product 
complaints).  DLA  assigned  a  lower  point  reduction  for  lab  test  failures  because 
DLA  laboratory  testing  often  focused  on  contractors  with  known  quality 
problems.  DLA  determined  that  ABVS  should  not  deduct  the  maximum  amount 
of  quality  points  from  a  contractor  that  DLA  tests  more  frequently  because  of 
quality  problems.  However,  in  January  1995  DLA  implemented  the  Product 
Verification  Program  which  led  to  changes  in  the  contractor  selection  process 
for  product  testing. 

Product  Verification  Pro^am.  Under  the  Product  Verification  Program, 

DLA  does  not  target  specific  contractors  for  laboratory  testing.  The  Product 
Verification  Program  conducts  laboratory  testing  of  contractors  selected  either 
through  random  sample  selection  of  products  or  as  the  result  of  a  directed 
product  laboratory  test.  Directed  tests  are  frequently  the  result  of  a  quality 
assurance  specialist’s  investigation  of  a  PQDR  to  determine  the  cause  of  the 
deficiency.  In  the  event  of  a  product  laboratory  test  failure  associated  with  a 
random  sample,  the  ICP  submits  a  PQDR  that  could  reduce  the  ABVS  quality 
rating  15  points  for  a  laboratory  test  failure  if  the  contractor  caused  the  product 
deficiency.  However,  DLA  personnel  do  not  issue  a  PQDR  for  a  directed 
laboratory  test  failure  associated  with  an  ongoing  PQDR  investigation.  Product 
Verification  Program  personnel  consider  the  primary  purpose  of  a  directed 
product  test  to  be  resolution  of  the  cause  of  the  product  deficiency,  not  data 
inputs  to  ABVS  quality  ratings  for  past  performance. 

Because  a  PQDR  is  not  submitted  to  record  a  directed  laboratoty  test  failure,  the 
ABVS  does  not  subtract  contractor  quality  points  for  the  test  failure.  As  a  result, 
a  contractor  does  not  receive  the  maximum  point  deduction  from  the  quality 
rating.  A  contractor  that  has  both  a  PQDR  and  random  test  failure  determined 
to  be  a  contractor  deficiency  could  receive  a  maximum  deduction  of  40  points. 
However,  if  a  contractor  has  only  a  PQDR  recorded  against  a  product  and  the 
contractor’s  product  was  not  randomly  selected  for  laboratory  testing,  the 
contractor  would  only  receive  a  maximum  deduction  of  25  points  for  product 
quality  deficiency.  Basically,  the  ABVS  quality  rating  relates  more  to  how  the 
ICP  detects  the  deficiency,  tiiat  is,  through  a  PQDR  or  laboratory  test  rather 
than  the  fact  that  a  product  deficiency  has  been  identified. 
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Contractors  can  receive  a  higher  quality  rating  than  warranted  for  a  product 
deficiency.  We  identified  753  contractors  witii  reduced  ABVS  quality  scores 
(packaging  not  included).  Of  the  753  contractors,  740  contractors  received 
quality  deductions  (25  points)  for  PQDRs;  however,  only  17  contractors  had 
quality  point  deductions  for  both  laboratory  product  test  failures  and  PQDRs 
(40  points).  The  ABVS  did  not  deduct  test  failure  rating  points  from  the  other 
723  contractors  because  those  contractors  were  either  not  chosen  for  a  random 
laboratory  test  or  the  ICP  did  not  record  the  test  failure  because  it  was  a 
directed  test  associated  with  a  PQDR.  Of  the  30  contractors  with  laboratory  test 
failures  only  (15  points),  13  contractors  still  received  25  points  for  product 
quality  because  the  ICPs  did  not  receive  a  PQDR  against  those  contractors. 


Figure  5.  Quality  Point  Deductions  Based  On  The  Number  Of  Known 

Contractor  Deficiencies 
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Combined  Quality  Rating.  Regardless  of  how  a  product  deficiency  is 
detected,  a  contractor’s  ABVS  quality  rating  should  reflect  that  deficiency. 

DLA  developed  the  ABVS  and  assigned  contractor  quality  points  to  reflect  the 
product  testing  procedures.  However,  as  the  DLA  testing  procedures  changed, 
contractors  became  less  likely  to  receive  point  deductions  from  both  PQDRs  and 
laboratory  test  failures.  DLA  should  eliminate  this  rating  inequity  and  use  only 
one  quality  rating  factor  of  40  points  for  either  a  PQDR  or  laboratory  test 
failure.  As  a  result,  ABVS  would  fully  reflect  a  contractor  product  deficiency 
and  not  reflect  the  method  of  deficiency  detection. 


Conclusion 


We  did  not  intend  to  identify  the  actual  effect  on  contracts  awarded  to 
contractors  based  on  ABVS  ratings.  However,  inaccurate  quality  ratings  of 
contractors  potentially  increases  die  risk  of  procuring  products  from  contractors 
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with  poor  past  performance.  Even  though  ABVS  ratings  do  not  determine 
eligibility  for  contract  award  or  technical  acceptability,  ratings  are  used  for 
comparison  of  past  performance  of  potential  contractors  with  the  Government. 

The  past  performance  of  a  contractor  who  has  delivered  late,  had  quality 
problems  or  failed  to  perform  in  the  past,  should  be  unfavorably  reflected  in 
ABVS  contractor  ratings.  A  contractor’s  low  score  translates  into  a  probable 
high  risk  of  future  late  deliveries,  but  it  will  also  reflect  hidden  costs  which  can 
result  in  lower  supply  availability,  increased  acquisition  and  procurement  lead 
times,  and  increased  costs  in  post  award  contract  administration. 


Recommendations  and  Management  Comments 


B.  We  recommend  that  the  Director,  Defense  Logistics  Agency  direct  the 
inventory  control  points  to: 

1.  Develop  quality  deficiency  reporting  procedures  to  ensure  that 
DLA  accurately  reports  contractor  quality  deficiencies  to  the  customer 
depot  complaint  system,  which  updates  the  Automated  Best  Value  System 
for  contractor  past  performance. 

2.  Notify  contractors  that  their  product  deficiency  will  he  reported 
in  the  Automated  Best  Value  System  quality  ratings  after  completing  an 
investigation. 

3.  Establish  a  single  quality  rating  for  the  Automated  Best  Value 
System  to  measure  contractor  product  quality  deficiency  and  product 
laboratory  test  failures. 

Defense  Logistics  Agency,  Comments.  DLA  concurred  with  all  of  the 
recommendations  stating  that  the  DLA  PQDR  PAT  will  revise  various  reporting 
codes  and  require  the  ICPs  to  report  quarterly  metrics  by  July  1998.  The  DLA 
PQDR  PAT  will  also  review  and  clarify  procedures  for  contractor  notification. 
DLA’s  policy  in  DLAD  4155.2  will  be  revised  to  include  the  recommended 
changes.  DLA  stated  that  by  February  15,  1998,  DLA  will  adopt  the 
recommendation  to  combine  laboratory  test  failures  and  product  deficiencies  for 
calculating  quality  ratings. 
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Part  II  -  Additional  Information 


Appendix  A.  Audit  Process 

Scope 


We  evaluated  the  DoD  PQDR  process  established  by  DLA  and  the  Military 
Departments,  which  reports  material  quality  conditions  across  DoD  Component 
organizations.  We  reviewed  the  System  for  product  deficiencies  and  product 
laboratory  test  failures  for  items  managed  by  DLA.  For  October  1,  1995, 
through  April  30,  1997,  we  reviewed  17,051  records  in  the  System  data  base 
that  were  recorded  for  deficiency  complaints  received,  closed,  and  the 
resolution  of  the  investigations.  We  reviewed  the  DLA  recoupment  records  for 
contractor  nonconforming  products  reported  in  the  System  data  base. 

We  reviewed  the  ABVS  that  rates  contractor  past  performance  to  determine  if 
contractor  quality  ratings  were  being  deducted  for  reported  product  deficiencies. 
We  evaluated  product  quality  deficiencies  and  product  laboratory  test  failures 
reported  in  the  System  data  base  which  were  investigated  and  determined  to  be 
caused  by  a  contractor  deficiency. 

We  reviewed  the  Joint  Logistics  System  Center’s  program  documentation  for 
the  Deficiency  Reporting  System  Program.  We  also  reviewed  the  Under 
Secretary  of  Defense  (Logistics),  Military  Departments,  and  DLA  decision  not 
to  implement  the  Deficiency  Reporting  System  Program  as  a  DoD-wide  product 
deficiency  information  system. 

We  did  not  use  statistical  sampling  procedures  for  this  audit. 


Audit  Period  and  Standards 


We  conducted  this  economy  and  efficiency  audit  from  January  1997  through 
October  1997,  in  accordance  with  auditing  standards  issued  by  the  Comptroller 
General  of  the  United  States,  as  implemented  by  the  Inspector  General,  DoD. 


Use  of  Computer-Processed  Data 


We  relied  on  DLA  computer-processed  data  contained  in  the  ABVS  and  the 
System  without  performing  tests  of  system  general  and  application  controls  to 
confirm  the  reliability  of  the  data.  We  did  not  establish  the  reliability  of  the 
data  because  it  was  not  within  the  scope  of  the  audit.  However,  we  did  examine 
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the  computer-processed  data  to  include  verifying  that  the  information  included 
the  fields  and  document  types  we  requested.  We  also  determined  that  the 
System  record  fields  contained  information  on  national  stock  number,  control 
number,  and  contract  number,  and  contractor  address  code. 


Management  Control  Program 


Requirement  for  Management  Control  Review.  DoD  Directive  5010.38, 
“Management  Control  Program,”  August  26,  1996,  requires  DoD  organizations 
to  implement  a  comprehensive  system  of  management  controls  that  provides 
reasonable  assurance  that  programs  and  administrative  and  operating  functions 
are  efficiently  and  effectively  carried  out  in  accordance  with  applicable  law  and 
management  policy. 

Scope  of  Review  of  Management  Control  Program.  We  evaluated 
management  controls  related  to  the  PQDR  program  described  in  DLAD  4155.2, 
“Quality  Assurance  Program  for  the  DLA  Inventory  Control  Points,”  DLAR 
4155.24,  “Product  Quality  Deficiency  Report  Program,”  and  DLAH  4140.4, 
“Customer  Depot  Complaints  System  Handbook.  ”  We  also  evaluated  Defense 
Industrial  Supply  Center,  Defense  Personnel  Support  Center,  Defense  Supply 
Center  Columbus,  and  Defense  Supply  Center  Richmond  procedures  for 
informing  contracting  officers  and  contractors  of  PQDRs.  In  addition,  we 
examined  management  control  procedures  to  ensure  that  quality  assurance 
personnel  knew  how  to  support  requests  for  recoupment  from  contractors  who 
supplied  defective  products.  We  also  reviewed  the  requirement  that  supervisors 
document  their  review  of  product  quality  deficiency  files  prior  to  closing  an 
investigation.  We  reviewed  the  DLA  results  of  self-evaluation  of  those 
management  controls  related  to  quality  and  PQDR  process. 

Adequacy  of  Management  Control  Program.  The  audit  identified  material 
control  weaknesses  as  defined  by  DoD  Directive  5010.38,  “Management 
Control  Program,”  August  26,  1996.  Management  controls  were  not  adequate 
to  ensure  that  correct  and  complete  product  quality  deficiency  information  was 
recorded  in  the  System.  Further,  management  controls  were  not  adequate  to 
ensure  that  the  System  information  for  contractor  nonconformance  was  being 
reported  to  the  Automated  Best  Value  System  for  quality  rating  of  contractor 
past  performance.  Recommendations  A.I.,  A.2.,  A. 3.,  B.I.,  and  B.2.  will 
correct  the  management  control  weaknesses.  A  copy  of  the  report  will  be 
provided  to  the  senior  official  responsible  for  management  controls  within  the 
Office  of  the  Director  for  DLA. 

Adequacy  of  Management’s  Self-Evaluation.  DLA  has  identified  the  PQDR 
program  and  process  as  management  control  assessable  units.  During  FY  1996, 
a  DLA  quality  management  review  was  performed  at  Defense  supply  centers  at 
Richmond,  VA;  Columbus,  OH;  and  Philadelphia,  PA.  The  quality 
management  review  did  not  identify  the  specific  material  management  control 
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weaknesses  identified  in  the  audit.  However,  the  review  did  identify  Quality 
Assurance  Specialist  personnel  not  performing  all  quality  functions  due  to  a 
large  volume  of  work;  and  inadequate  Quality  Assurance  Specialist  training 
required  to  perform  their  job  functions. 
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IG,  DoD  Report  No.  94-079,  “DoD  Component  Implementing  Action  Plans 
for  Improving  the  Quality  of  Spare  Parts,”  April  12, 1994,  found  that  the 
DoD  Components  need  to  establish  additional  initiatives  for  feedback, 
reporting,  data  exchange,  and  definitions  to  the  PQDR  Program.  As  a  result  of 
recommendations,  the  Army,  Air  Force  and  Navy  established  policies  to  reissue 
and  update  action  plans  on  a  more  timely  basis.  DLA  management  developed 
and  published  an  updated  action  plan  in  December  1993. 

IG,  DoD  Report  No.  93-066,  “Recoupments  for  Quality  Defects,” 

March  10,  1993,  stated  that  the  Defense  Logistics  Agency  Defense  Supply 
Centers  did  not  perform  complete  quality  assurance  investigations  because 
quality  assurance  specialists  frequently  curtailed  PQDR  investigations  without 
validating  quality  deficiencies  through  the  supplier.  As  a  result  of  the 
recommendations,  DLA  issued  a  policy  memorandum  requiring  QASs  to  meet 
certain  objectives  in  a  PQDR  investigation.  Also,  DLA  implemented 
procedures  to  perform  laboratory  tests  as  a  means  to  support  a  product 
deficiency  investigation. 

IG,  DoD  Report  No.  92-099,  “Quality  Assurance  Actions  Resulting  from 
Electronic  Component  Screening,”  June  8,  1992,  described  problems  with 
the  collection,  distribution,  and  use  of  quality  deficiency  information  in  DoD. 
As  a  result  of  the  recommendations,  the  Director  of  Defense  Procurement 
requested,  and  DLA  officials  agreed  to  identify  problem  products  and  product 
lines/suppliers  to  address  nonconforming  products,  and  the  policy  covering 
recoupments  for  products  with  major  nonconformances. 

IG,  Air  Force  Audit  Agency,  Report  No.  95061021,  “Deficiency  Reporting 
Management,”  November  28,  1995,  reported  that  the  PQDR  program  was 
cumbersome.  Based  on  interviews,  no  more  than  10  to  15  percent  of  all 
deficiencies  were  reported  on  PQDRs.  The  report  describes  problems  with 
training  and  guidance  of  base  personnel.  As  a  result  of  the  recommendations, 
the  Air  Force  Materiel  Command  is  implementing  a  new  training  program 
explaining  PQDR  procedures. 

Office  of  Internal  Review,  DSCC  Columbus,  “Review  of  Product  Quality 
Deficiency  Reports  (PQDR)  (In  Land  Application),”  June  20,  1997,  found 
that  documentation  and  notification  to  the  customer  needed  to  be  improved  and 
a  more  effective  communication  system  established.  As  a  result  of  tiie 
recommendations,  management  devised  management  controls  for  tracking 
complaints  by  national  stock  numbers.  Also,  management  agreed  to  sample 
PQDR  reports  to  verify  customer  satisfaction. 

Office  of  Internal  Review,  DSCR  Richmond,  “Recoupment  Actions  Taken 
on  Contractor  Noncompliance  Product  Quality  Deficiency  Reports,  Audit 
Control  No.  J9610,”  January  24,  1997,  found  that  recoupment  actions  were 
being  taken  and  were  generally  adequate.  However,  a  lack  of  communication 
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between  quality  personnel  and  contracting  personnel  was  generally  the  reason 
when  no  recoupment  action  was  taken.  As  a  result  of  the  recommendations, 
management  modified  the  process  of  reporting  recoupment  actions  and 
reemphasized  the  applicable  guidance  needed  for  successful  recoupments. 

Office  of  Internal  Review,  DISC  Philadelphia,  “Audit  of  Compliance  vdth 
the  DLA  Action  Plan  for  Continuously  Improving  the  Quality  of  Spare  and 
Repair  Parts  in  the  DoD  Logistics  System,”  May  20,  1992,  found  that  the 
internal  coordination  and  related  actions  after  the  receipt  of  information  on 
nonconforming  products  was  adequate.  Additionally,  the  procedures  for 
ensuring  that  the  performance  of  contractors  was  being  factored  into  the  source 
selection  process  were  not  yet  implemented  because  of  a  DLA  Headquarters 
directed  change. 
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At  the  present,  each  DoD  Component  has  its  own  information  system  for 
reporting,  processing  and  tracking  the  progress  of  deficiency  reports.  The 
DoD-wide  Deficiency  Reporting  System  Program  was  initiated  to  replace 
existing  DoD  Component  product  legacy  deficiency  systems  and  to  automate  the 
deficiency  reporting  process  and  provide  users  with  a  means  to  perform  eight 
primary  business  functions.  The  formal  program  description  defines  these 
business  functions  as: 

o  Identify  a  deficiency 

o  Originate  a  customer  complaint 

o  Screen  the  complaint 

o  Investigate  the  deficiency 

o  Coordinate  disposition  of  the  discrepant  material 

o  Resolve  the  deficiency 

o  Analyze  performance 

The  Deficiency  Reporting  System  included  PQDRs  but  also  several  other  types 
of  deficiency  reports,  such  as  Reports  of  Discrepancy,  Medical  Complaints,  and 
Transportation  Discrepancy  Reports. 

In  June  1992,  the  Deficiency  Reporting  System  Program  development  started 
with  the  Army  tasked  to  develop  and  implement  the  program,  with  program 
deployment  planned  for  September  1994.  The  program  was  transferred  from 
the  Army  to  the  DoD  Joint  Logistics  Systems  Center  in  1994. 

In  October  1996,  the  Assistant  Deputy  Under  Secretary  of  Defense  (Logistics 
Business  Systems  and  Technology  Development)  issued  a  memorandum 
terminating  all  future  software  development  and  deployment  activities  for  the 
Deficiency  Reporting  System  Program.  By  February  1997,  about  $37  million 
had  been  spent  on  program  development. 

During  1997,  the  Deficiency  Reporting  System  underwent  Army  software  tests 
(alpha  version  Ic)  for  possible  Army  adoption.  However,  the  software  tested 
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did  not  include  all  the  capabilities  originally  planned  and  the  resulting  program 
capability  included  only  PQDRs  and  Report  of  Discrepancy  for  customer 
complaints. 

Current  Status.  In  September  1997,  the  Army  completed  Deficiency 
Reporting  System  testing  of  the  version  Ic  software.  The  Army  has  requested 
that  the  Joint  Logistics  Systems  Center  release  the  software  and  supporting 
documentation  for  future  fielding  if  Army  funds  become  available.  DLA, 
Navy,  and  Air  Force  will  continue  using  their  current  legacy  information 
systems  for  product  deficiency  reports  and  they  do  not  plan  on  using  the 
Deficiency  Reporting  System’s  software  in  the  fiimre. 
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Each  DLA ICP  provided  System  records  in  electronic  format.  The  System 
records  were  document  types  0  and  1  (PQDRs);  Type  2  (Depot  Returns);  and 
Type  4  (Laboratory  Tests)  recorded  from  October  1,  1995  through  April  30, 
1997.  j^ch  record  included  the  following  data  fields. 

Report  Control  Number  -  the  Center’s  control  number  for  the  complaint. 

Receive  Date  -  the  date  the  complaint  was  received  and  recorded  into  the 
System. 

Close  Date  -  the  date  the  center  has  completed  all  actions  on  resolution  of  the 
complaint  and  recorded  it  into  the  System. 

Document  Type  Code  -  the  code  determines  if  the  record  is  a  Category  I  or  II 
PQDR. 

Discrepancy  Code  -  a  code  summarizing  the  reason  for  the  complaint. 

Cause  Code  -a  code  summarizing  the  cause  of  the  complaint. 

Disposition  Code  -a  code  summarizing  the  settlement  of  the  complaint. 

Correction  Code  -a  code  summarizing  the  action  taken  to  prevent  the 
reoccurrence  of  the  complaint. 

National  Stock  Number  -  the  stock  number  assigned  to  identify  a  product  for 
procurement. 

CAGE  Code  -  Commercial  and  Government  Entity  code. 

DoDAAC  Reporting  -  Department  of  Defense  Activity  Address  Code. 
Contract  Number  -  the  contract  the  nonconforming  item  was  procured  from. 
Quantity  Reported  -  the  quantity  listed  on  the  complaint. 

Unit  Cost  -  the  unit  cost  of  the  nonconforming  item. 
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An  action  point  is  the  focal  point,  identified  within  each  DoD  Component, 
responsible  for  resolution  of  a  reported  product  quality  deficiency,  including  the 
necessary  collaboration  with  support  points  (usually  Defense  Contract 
Management  Command).  Only  an  action  point  is  authorized  to  transmit  a 
deficiency  report  across  Component  lines.  The  Defense  Logistics  Agency 
Regulation  4155.24/AR  702  702-7/SECNAVINST  4855.5A/AFR  74-6, 
“Product  Quality  Deficiency  Report  Program,”  October  7,  1996,  (DRAFT) 
defines  the  action  point’s  three  primary  fimctions  in  the  PQDR  process. 


General  Area  of  Responsibility 


o  Investigates,  resolves,  and  responds  in  a  timely  manner  and  coordinates 
with  cognizant  Defense  supply  center  elements  to  ensure  that  other  functional 
DoD  Component  organizations  are  kept  abreast  of  actions  taken  on  PQDRs. 

o  Takes  corrective  action  on  reported  product  defects  and  on  the  cause  to 
preclude  recurrence,  issues  immediate  notification  to  users  of  the  product  if  the 
deficiency  is  serious. 

o  Provides  instructions  for  disposition  and  credit  allowances  to  DoD 
Components. 

o  Analyzes  and  evaluates  product  deficiencies  to  detect  trends  of  poor 
quality  products. 

o  Identifies  contractors  who  provide  defective  products. 

o  Shares  quality  history  information  with  the  other  Defense  supply  center 
elements  and  DoD  Components. 


Investigations  of  PQDRs 


0  Reviews  the  contract,  complaints,  item  and  contractor  history, 
technical  data,  laboratory  tests,  and  inventory  balances. 

o  Manages  and  coordinates  the  examination  of  the  defective  product. 
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Appendix  E.  Product  Quality  Deficiency  Report  Action  Point  Responsibilities 


o  Determines  the  need  for  an  investigation  by  the  support  point  (usually 
the  Defense  Contract  Management  Command)  or  determines  the  need  for  a 
quality  system  management  visit  by  the  action  point. 

o  Evaluates  the  total  investigation  results  and  determines  if  the  PQDR  is 

valid. 

0  Takes  the  appropriate  corrective  action  if  the  PQDR  is  valid. 


Corrective  Actions  for  PQDRs 


o  Screens  on-hand  inventory  to  identify  and  separate  defective  products. 

o  Acts  to  preclude  recurrence  by  recommending  specification  changes, 
changes  the  contract  technical  data  file,  issues  quality  assurance  letters  of 
instruction  to  inspection  activities,  advises  contracting  officers  of  an  adverse 
quality  history,  and  notifies  contractors  of  the  PQDR. 

o  Initiates  recoupment  actions  against  suppliers  of  defective  products. 

o  Coordinates  reclassification  with  supply  managers  and,  if  needed, 
coordinates  the  disposition  of  inventory. 

o  Notifies  inspection  activities  of  future  inspections,  contracting 
activities  of  future  contracts,  and  requisitioning  activities  of  quality  deficiencies. 


31 


Appendix  F.  Comparison  by  Supply  Centers  of 
the  Use  of  Cause  Codes 

Percentage  Based  on  16,898  Records 
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Appendix  G.  Comparison  by  Supply  Centers  of 
the  Disposition  of  Other/Does  Not  Apply 
Cause  Code 


Percentage  Based  on  7,691  Records 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Acquisition  and  Technology) 
Deputy  Under  Secretary  of  Defense  (Logistics) 

Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Assistant  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management) 
Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  National  Imagery  and  Mapping  Agency 
Inspector  General,  Defense  Intelligence  Agency 


Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 
Office  of  Federal  Procurement  Policy 
Small  Business  Administration 
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Appendix  H.  Report  Distribution 


Non-Defense  Federal  Organizations  (Cont’d) 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional  committees 
and  subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  Government  Management,  Information  and  Technology, 
Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal  Justice, 
Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
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Part  in  -  Management  Comments 


Defense  Logistics  Agency  Comments 


DEFENSE  LOGISTICS  AGENCY^ 
HEADQUARTERS 

8725  JOHN  J.  KINGMAN  ROAD.  SUITE  2533 
FT.  BELVOIR.  VIRGINIA  22060-6221 


JAN  2  2  1998 


MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING, 
DEPARTMENT  OF  DEFENSE 


SUBJECT:  Draft  Report  on  Defense  Logistics  Agency  Product  Quality  Deficiency  Program, 

7CF-0027 


Enclosed  is  our  response  to  your  request  of  25  November  1997.  Should  you  have  any  questions 
please  contact  LaVaeda  Coulter,  767-6261  or  Sharon  Entsminger,  767-6267. 


Enel 


cc: 

DLSC-LE 

DSCC-DI 

DISC-Dl 

DSCR-DI 

DPSC-OSPP 


IN  REPLY 
REFER  TO 

DDAI 


FMMrai  Hccycitno  Progrim  on  Rocydad  Papof 


Defense  Logistics  Agency  Comments 


SUBJECT;  Defense  Logistics  Agency  Product  Quality  Deficiency  Program,  7CF-0027 

FINDING  A;  Quality  Deficiency  Reporting  (Page  5  of  Draft  Report) 

DLA  COMMENTS:  Concur.  DLA  agrees  that  additional  actions  arc  needed  to  better  identify 
the  causes  of  reported  product  deficiencies.  DLA  will  initiate  a  Process  Action  Team  (PAT)  to 
address  the  recommendations  for  corrective  action.  The  PAT  will  modify  the  customer  depot 
complaint  system,  as  required,  to  enhance  various  reporting  codes.  The  Agency  will  request  that 
the  Defense  Inventory  Control  Points  report  quarterly  via  the  Materiel  Management  Monthly 
Management  Reviews  on  product  deficiency  cause,  correction,  and  disposition  codes  for  closed 
investigations.  Agency  policy  in  DLAD  4155.2  will  be  revised  to  incorporate  recommended 
changes. 

RECOMMENDATION  A;  Recommend  that  the  Director,  Defense  Logistics  Agency: 

1 .  Modify  the  customer  depot  complaint  system  to  require  the  Commercial  and  Government 
Entity  Code  as  a  mandatory  entry  for  closing  a  complaint  whenever  the  record  includes  the 
contract  number. 

2.  Modify  the  customer  depot  complaint  system  to  require  the  DoD  Activity  Address  Code  as 
a  mandatory  entry  for  closing  a  product  quality  deficiency  investigation. 

3.  Clarify  proper  use  of  the  other/does  not  apply  cause  code  for  customer  deficiency 
complaints. 

4.  Eliminate  the  use  of  special  inspection  as  a  cause  code  for  customer  deficiency  complaints. 

5.  Revise  the  customer  depot  complaint  system’s  cause,  disposition  and  action  codes  and 
replace  them  with  more  descriptive  codes  for  deficiency. 

6.  Require  the  defense  inventory  control  points  to  report  quarterly  metrics  on  product  quality 
deficiency  cause,  correction,  and  disposition  codes  for  closed  investigations.  The  report  should 
also  include  information  on  initially  missing  contract  numbers.  Commercial  and  Government 
Entity  Codes,  and  DoD  Address  Activity  Codes. 

DLA  COMMENTS:  Concur.  DLA  agrees  that  additional  actions  are  needed  to  better  identify 
the  causes  of  reported  product  deficiencies.  DLA  will  initiate  a  Process  Action  Team  (PAT)  to 
address  the  recommendations  for  corrective  action.  The  PAT  will  modify  the  customer  depot 
complaint  system,  as  required,  to  enhance  various  reporting  codes.  The  Agency  will  request  that 
the  defense  inventory  control  points  report  quarterly  via  the  Materiel  Management  Monthly 
Management  Reviews  on  product  deficiency  cause,  correction,  and  disposition  codes  for  closed 
inv  estigations.  Agency  policy  in  DLAD  4155.2  will  be  revised  to  incorporate  recommended 
changes. 


Defense  Logistics  Agency  Comments 


PAGE  2 

SUBJECT:  Defense  Logistics  Agency  Product  Quality  Deficiency  Program,  7CF-0027 

DISPOSITION:  Ongoing.  ECD:  July  31, 1998 

Initial  Process  Action  Team  meeting  February  1 998 

Develop  draft  codes  March  1998 

Draft  metrics  March  1 998 

Coordinate  and  finalize  codes/metrics  May  1998 

Draft  policy  letter  for  coordination  May  1 998 

PAT  Meeting  to  finalize  actions  June  1 998 

Issue  interim  policy  letter  July  1998 

Initiate  System  Change  Request  July  1998 

FINDING  B:  Automated  Best  Value  System  (Page  15  of  Draft  Report) 

DLA  COMMENTS:  We  concur  with  the  finding  as  written.  The  ABVS  strives  to  obtain  the 
most  accurate  quality  ratings  for  its  contractors.  Participating  in  this  audit  provided  an 
opportunity  for  the  procurement  and  quality  communities  to  work  together  to  improve  our 
methodology  for  obtaining  quality  ratings.  The  ABVS  relies  on  the  information  provided 
through  the  DLA  automated  systems  such  as  the  customer  depot  compliant  system.  The 
recommendation  set  forth  in  this  finding  to  establish  a  single  quality  rating  for  the  ABVS  to 
measure  contractor  product  quality  deficiency  and  product  laboratory  test  failures  has  been 
implemented  and  will  become  effective  with  the  February  1 998  run  of  the  ABVS. 

RECOMMENDATION  B;  Recommend  that  the  Director,  Defense  Logistics  Agency  direct  the 
inventory  control  points  to: 

1.  Develop  quality  deficiency  reporting  procedures  to  ensure  that  DLA  accurately  reports 
contractor  quality  deficiencies  to  the  customer  depot  complaint  system,  which  updates  the 
Automated  Best  Value  System  (ABVS)  for  contract  past  performance. 

Concur.  Recommendation  A  addresses  improvements  to  accurate  reporting  of  contractor  quality 
deficiencies  in  the  contractor  depot  complaint  system.  The  DLA  PQDR  PAT  will  revise  various 
reporting  codes  and  require  the  defense  inventory  control  points  to  report  quarterly  metrics  by 
July  1998. 

2.  Notify  Contractors  that  their  product  deficiency  will  be  reported  in  the  ABVS  quality  ratings 
after  completing  an  investigation. 


Defense  Logistics  Agency  Comments 


PAGES 

SUBJECT:  Defense  Logistics  Agency  Product  Quality  Deficiency  Program,  7CF-0027 


Concur.  DL  A’s  quality  assurance  component  has  the  responsibility  to  notify  contractors  of 
a  deficiency.  Once  a  contractor  is  notified  of  a  deficiency  this  deficiency  is  flagged  in  the  ABVS 
system  through  the  challenge  program  and  the  contractor  is  given  an  opportunity  to  challenge  the 
deficiency.  It  is  only  after  this  is  done  that  a  quality  score  is  given.  The  DLA  PQDR  PAT  will 
review  and  clarify  procedures  for  contractor  notification.  Agency  policy  in  DLAD  4155.2  will  be 
revised  to  incorporate  recommended  changes. 

3.  Establish  a  single  quality  rating  for  the  ABVS  to  measure  contractor  product  quality 
deficiency  and  product  laboratory  test  failures. 

Concur.  Scheduled  completion  of  this  recommendation  is  on  or  about  February  1 5,  1998. 
Currently,  the  ABVS  Quality  Rating  consists  of  three  categories:  product  deficiencies,  lab  tests, 
and  packaging  deficiencies.  The  decision  has  been  made  as  a  result  of  this  audit  to  no  longer 
consider  lab  tests  through  a  separate  system,  therefore  making  them  not  distinguishable  firom 
other  deficiencies.  Now  when  deficiencies  are  identified  as  a  result  of  lab  test  failures,  they  will 
either  be  classified  as  a  product  or  packaging  deficiency  and  entered  into  the  appropriate  ABVS 
category. 

Beginning  on  or  about  February  15, 1998,  the  ABVS  Quality  Rating  will  consist  of  two 
categories:  product  deficiencies  (this  category  will  include  both  laboratory  test  failures  and 
product  complaints  -  both  captured  in  the  Customer  Depot  Complaint  System)  and  packaging 
deficiencies.  Because  of  this  change  in  the  relative  weighting  of  the  categories  a  notice  has  been 
sent  out  to  all  contractors  that  within  the  Quality  Rating  their  ABVS  score  may  change.  Scores 
which  have  been  notoriously  high  will  now  reflect  a  more  realistic  quality  score. 

DISPOSITION:  Ongoing.  ECD:  July  31, 1998 
Revise  ABVS  Quality  Ratings 
Initial  PAT  meeting 

Redraft  contractor  notification  procedures 
PAT  Meeting  to  finalize  procedures 
Issue  interim  policy  letter 

ACTION  OFFICER:  Lynn  Harris/Althea  Norman 
REVIEW:  John  A.  Marx,  Col,  USA 
COORDINATION:  LaVaeda  Coulter,  DDAI,  767-6267 

DLA  APPROVAL:  VOn 


February  1998 
February  1998 
March  1998 
June  1998 
July  1998 
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Audit  Team  Members 


This  report  was  prepared  by  the  Contract  Management  Directorate,  Office  of 
the  Assistant  Inspector  General  for  Auditing,  DoD. 

Paul  J.  Granetto 
Terry  L.  McKinney 
Michael  H.  Claypool 
Chuck  J.  Chin 
Shawn  L.  James 
Harold  R.  Tollefson 
Joseph  P.  Bucsko 
Yolanda  C.  Watts 
Ana  M.  M5rrie 


